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Abstract 

This paper presents the rationale and research base for a reading competency model designed to 
guide the development of cognitively based assessment of reading comprehension. The model 
was developed from a detailed review of the cognitive research on reading and learning and a 
review of state standards for language arts. A survey of the literature revealed three key areas of 
reading competency: prerequisite reading skill, model building skill, and applied comprehension 
skill. Prerequisite reading skill is the ability to read text accurately and fluently. Model building 
skill is the ability to construct meaning from either decoded text or spoken language. Applied 
comprehension skill is the ability to use and apply the information contained in text for some 
particular purpose. The framework is discussed in terms of 7 key principles that have 
implications for the design of a modern assessment of reading. 
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Introduction 



For the better part of the 20th century, the United States has been recognized for its social 
mission, innovation, and economic prosperity. However, with the increase in global 
communication and global marketing, the United States is now facing enhanced competition 
from countries such as India and China. America’s continued economic prosperity and its 
position in the global economy are also threatened by three internal problems: a changing 
economy, divergent skill distributions, and demographic trends leading to a more diverse 
population (Kirsch, Braun, Yamamoto, & Sum, 2007). 

America’ s changing economy has created a demand for a more highly skilled labor force, 
as the jobs that will be created in the future, more and more, will require a college degree. 
According to Kirsch et al. (2007), however, the United States is currently experiencing a general 
deficit in critically needed skill areas such as reading and math, and these achievement gaps have 
remained stable over time. Changing demographics are also a contributing factor because 
populations that traditionally had relatively low levels of education and literacy are increasing 
most rapidly. As Kirsch et al. argued, these factors, if left unchecked, threaten to dramatically 
change the U.S. position in the world economy. Consequently, Kirsch et al. referred to these 
trends as America’ s perfect storm. 

What, then, can be done to help prevent America’s perfect storm from happening? At 
least part of the answer depends upon policy makers’ ability to promote educational reforms that 
will enable all students to acquire the reading and higher level thinking skills needed to succeed 
in the 21st century. Over the past 10 years, federal policy makers have advocated change though 
the vehicle of high stakes testing. Federal legislation such as the No Child Left Behind (NCLB) 
Act (2002) is intended to promote change by making schools accountable for the reading and 
math skills of all of their students. While the effects of high stakes testing are highly contested 
(McNeil, 2000; Nicholas & Berliner, 2007), it is likely that the NCLB legislation, probably in 
modified form, will continue to influence education in the near future. 

Education reform has also gained traction at the state level as states have begun to wrestle 
with the effects of local economic changes such as the loss of manufacturing jobs and the 
outsourcing of technical work to China and India. State reform efforts have included a greater 
emphasis on science and technology education and the development of new curricular standards 
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that explicitly address the kinds of high level skills (e.g., critical thinking, problem-solving) 
valued by modern-day business leaders. 

A comprehensive system designed to address the concerns noted previously is described 
in Bennett and Gitomer (in press). This new approach is called cognitively based assessment of 
for and as learning (CBAL). The CBAL approach involves first developing a cognitive model of 
domain competency in the targeted content area and then using that model to design tasks and 
activities that synergistically address three crucial aspects of educational practice: accountability 
assessment, formative assessment, and professional support. Successful implementation requires 
five key innovations. 

1. Individual tasks are designed to assess the integrated use of the knowledge and skills 
specified in the competency model and to target aspects of students’ performances 
that cut across different curricular units. 

2. Administration of the accountability assessment is distributed across multiple testing 
sessions spaced throughout the school year so that (a) the importance of any one 
assessment and occasion is diminished, (b) tasks can be more complex and integrative 
because more time is available for assessment in the aggregate, 1 and (c) test results 
can be provided to teachers while time is still available to take appropriate 
instructional action. 

3. Tasks and activities are designed to be learning experiences in and of themselves so 
that a strategy of teaching to the test is then an appropriate instructional activity. 

4. Because all of the assessments are based on a common competency model, results 
from the various accountability assessments can serve as prior information for 
subsequent formative and diagnostic assessments. 

5. Assessments consist primarily of computer-delivered constructed-response tasks 
scored via a combination of technologies, including both computer-assisted human 
scoring and automated scoring. 

This paper describes an application of the CBAL approach to the domain of reading 
comprehension. The purpose of the paper is to describe the rationale and research base 
underlying the CBAL reading framework. The paper first provides an overview of the CBAL 
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reading competency model, followed by a discussion of the research base and key cognitive 
principles that were used to create the CBAL reading framework and competency model. The 
paper then concludes with a brief summary. 

The Cognitively Based Assessment of, for, and as Learning (CBAL) Reading Framework 

The CBAL reading competency model was created by performing a detailed review of 
the cognitive research on reading and a survey of state reading standards. The starting point for 
the model was based on the framework contained in the Rand Reading Study Group (RRSG) 
report titled Reading for Understanding Toward an R&D Program in Reading Comprehension 
(RRSG, 2002). The RRSG identified three important aspects of reading comprehension: the 
reader, the text, and the reading activity. Characteristics of the reader include the cognitive 
processes, skills, strategies, and knowledge that enable the reader to extract meaning from the 
text. Characteristics of the text include the collection of linguistic and content features that 
contribute to the ease or difficulty of extracting meaning from text. Characteristics of the activity 
include the purpose, goals, tasks, and the expected outcome(s) of reading. The interactions 
between the reader, the text, and the activity occur in the context of the larger sociocultural 
environment. 

Reading is a complex process that requires the coordinated use of a large number of 
skills. In a broad sense, successful reading requires three general classes of skills: prerequisite 
reading skill, model building skill, and applied comprehension skill (see skills branch in Figure 1 
and in the appendix for a detailed description of the skills branch). Prerequisite reading skill is 
the ability to accurately read text at a rate that is grade-level appropriate. It includes the ability to 
recognize familiar words (e.g., house) and the ability to decode unfamiliar words. This latter 
ability is particularly important for struggling readers as many printed words are unfamiliar. It is 
also important for on grade-level readers since many technical content terms are unfamiliar (e.g., 
mitosis), not to mention place and person names (e.g., Somalia, Shrek) and product labels (e.g., 
Zantec). Prerequisite reading skill also includes the ability to read text fluently with appropriate 
expression and intonation (i.e., prosody). It is important to note that prerequisite reading skills 
are bound to printed text and directly represent reading ability in its literal sense. Reading ability 
in this context means the ability to decode and accurately recognize printed text, not the ability to 
comprehend text (see model building skill in the next paragraph). In short, prerequisite reading 
skill is the foundational set of skills that enables readers to advance to the comprehension stage 
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of model building. Without adequate prerequisite reading skill, comprehension of printed text 
will suffer. 




Figure 1. The Cognitively Based Assessment of, for and as Learning (CBAL) reading 
competency model. 
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Model building skill is the collection of abilities that allows one to construct meaning 
from either decoded text or spoken language. This skill set includes all of the skills needed to 
construct meaning from words (vocabulary), sentences, paragraphs, and the overall discourse 
structure of text. Model building involves the ability to locate and retrieve information (literal 
comprehension) as well as the ability to infer and generalize unstated relationships within text. 
Both the literal and inferential levels of text processing help the reader to construct a mental 
model of a text’s meaning. A mental model is a structured representation of the literal and 
implied meaning of text. It includes the ability to chunk, organize, and summarize information. 
While model building skill can occur in the context of printed text, it is not necessarily tied to 
print; model building skill can operate in contexts other than print, such as spoken language. 

Applied comprehension skill is the ability to use the information contained in text or 
spoken language for some particular purpose. Applied comprehension involves going beyond the 
literal and inferential interpretation of text or spoken language in order to use the information to 
achieve a particular goal (e.g., solve a problem, make a decision, create a presentation or Web 
site). Applied comprehension in the CBAL model is broken down into three types of reading: 
reading that requires integrating and synthesizing information from multiple sources; reading 
that involves reasoning, explaining, and generating explanations by integrating new information 
with relevant background knowledge; and reading that requires application of critical thinking 
skills to evaluate text contents (evaluate/critique).” Applied comprehension can occur in the 
context of printed text (reading) or in the context of spoken language; it is not bound to print. 
Moreover, applied comprehension may go beyond general definitions of reading comprehension 
by extending the definition to cover areas of critical thinking and critical reasoning. 

The competency model also includes the strategy branch to guide assessment design by 
ensuring tasks are included that call upon these strategies. The strategy branch is important 
because research has shown positive effects of reading strategy use on comprehension 
(McNamara, Ozuru, Best, & O’Reilly, 2007). Strategies are deliberate, conscious, effortful 
actions that successful readers implement to repair breaks in comprehension and to move 
understanding from a shallow level to a deeper level. In the CBAL framework, reading strategies 
are classified into one (or more) of the following categories: strategies that can help readers 
overcome deficiencies in prerequisite reading skill (e.g., previewing the text), strategies that can 
help readers build better (i.e., more coherent) mental models (e.g., representing text via tables or 
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other graphic organizers), and strategies that can help readers master the applied comprehension 
skill (e.g., adopt a critical stance, evaluate source/claims, looking for biases, etc.). Note that these 
classifications support the goal of integrating assessment and instruction by providing a link to 
the skill categories targeted in the assessment. In this way, the competency model provides a 
framework for defining assessment tasks that yield high quality evidence about targeted skills 
while simultaneously supporting both teaching and learning. 

As is shown in Figure 1, the competency model also includes a text branch, which is the 
set of conventions and characteristics students must be conversant with if they are to be 
successful in making meaning from text. The text branch of the competency model represents the 
set of skills that students must have and which are accounted for in assessment design by 
including text with varying characteristics. Since many important text characteristics interact 
significantly with genre (see Sheehan, Kostin, & Futagi, 2008a, 2008b) distinct text difficulty 
models are employed for each of three targeted genres: literary text, informational text, and 
persuasive text. For more detailed information about these models, please see Sheehan et al. 

Seven- Key Cognitive Principles and Their Implications for Creating a Cognitively Based 
Assessment of, for, and as Learning (CBAL) Accountability Assessment for Reading 

The CBAL reading framework can be summarized in terms of seven key cognitive 
principles. This section provides a description, rationale, and research base for each principle. 
Implications for the design of CBAL reading accountability assessments that provide valid and 
reliable evidence about examinees’ mastery status on targeted skills are also presented. 

Provide a Realistic Purpose for Reading 

Reading is an inherently purpose-driven activity (Alderson, 2000). Every day, people 
read for a variety of purposes, including to learn new information, to solve problems, to facilitate 
decision making, and to be entertained. Having a purpose for reading is important for several 
reasons, not the least of which is that the purpose helps to define the goals and motivation for 
reading. In essence, the reader’s purpose provides a road map for determining which pieces of 
information are, and are not, important to focus on (Pressley, 2000). In other words, the purpose 
allows the reader to be selective in his or her reading. For example, if the goal of reading were to 
simply find out the year in which a specific historical event occurred, understanding (or even 
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reading) the sequence of actions that led up to that event would not be necessary as it would not 
directly help the reader achieve his or her specific goal. 

Establishing a purpose for reading also helps to define the reader’s standards of 
coherence (Finderholm, Virtue, Tzeng, & van den Broek, 2004; van den Broek, Forch, 
Linderholm, & Gustafson, 2001; van den Broek, Risden, & Husebye-Hartman, 1995). The term 
standards of coherence refers to the level of comprehension that a reader strives to achieve. A 
reader’s standards of coherence dictate how deeply a text is processed and the degree of 
uncertainty deemed acceptable. Readers who choose to have a high level of coherence must 
expend additional effort (e.g., elaborate, draw inferences) to ensure that the intricate details of a 
text are understood. Conversely, readers who establish a low standard of coherence are more 
likely to tolerate gaps in understanding. 

Some evidence suggests that focusing a reader’s attention on specific portions of a text 
can enable purpose-driven interest, which in turn promotes recall of the focused sections of the 
text (Schraw & Dennison, 1994); thus, providing a purpose for reading can increase interest 
and improve memory for text. This is an important result, as research has shown that intrinsic 
reader interest is positively related to reading comprehension (Cox & Guthrie, 2001; Wang & 
Guthrie, 2004). 

It is no surprise, then, that research has shown that reading for different purposes can lead 
to differential processing of text (van den Broek et al., 2001) and differential use of reading 
strategies (Braten & Samuelstuen, 2004). For instance, people who are asked to read for the 
purposes of studying for a test draw more coherence-based inferences than people who are asked 
to read for entertainment. Similarly, people are more likely to produce associations and 
evaluations when reading for entertainment than when reading to prepare for a test (van den 
Broek et al., 2001). 

Despite the wealth of research on the impact of reader purpose on comprehension, many 
current assessments do not provide any purpose for reading other than to answer as many 
questions as possible in the allotted timeframe. As Rupp, Feme, and Choi (2006) put it, “When 
people read in a nontesting context, they do not answer multiple choice questions in their heads” 
(p. 441). The take-home message then is that the contexts underlying most traditional reading 
assessments are somewhat arbitrary and artificial. Outside of the lab and testing situations, 
people read for specific and meaningful purposes. Perhaps the most important conclusion to 
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draw from the literature is that establishing a purpose for reading helps to improve reader 
interest, which in turn can improve comprehension. 

The research summarized previously has implications for assessment design. In 
particular, it suggests that reading assessments should provide a goal-directed purpose for 
reading that is instantiated in the context of a meaningful scenario. The adjective meaningful 
implies that scenarios should be illustrative of the age-appropriate situations in which students 
would realistically be expected to read texts to achieve a particular purpose. Recent assessments 
that utilize scenarios in this way include the Collegiate Learning Assessment produced by the 
Council for Aid to Education and the iSkills™ test produced by ETS. 

Scenario-based reading assessments contextualize the reading activity by first defining a 
meaningful purpose for reading and then presenting a collection of texts related to that purpose. 
Assessment scenarios should strive to achieve three aims: (a) provide a general reason for 
reading the particular collection of texts presented (e.g., to make a decision on whether to adopt a 
wind power as a source of energy in your community), (b) constrain the activity by providing 
specific subgoals (e.g., consider community-specific needs such as location and availability of 
wind), and (c) clarify expectations regarding the culminating task demands (e.g., create a Web 
site that informs the community about what wind power is, describes the pros and cons of 
building a wind farm, and provides possible solutions that address the drawbacks of using wind 
power as a source of energy in your community). 

Satisfaction of these aims may also lead to additional benefits. For example, providing a 
reason for reading specific collection texts may also increase motivation and help students align 
their processing, resources, and strategies to match the task at hand. Providing subgoals allows 
the assessment designer to measure the ability to follow directions. Providing an expectation for 
what is to be produced should help increase student motivation because the reading activity is 
expected to produce some tangible result. Resulting scenario-based assessments not only provide 
a cohesive purpose for reading, but they also set the stage for the next two principles: integration 
and evaluation. 

Measure the Ability to Integrate and Synthesize Information From Multiple Related Texts 

Goldman (2004) noted that, because many traditional off-the-shelf reading assessments 
are designed to measure comprehension of individual passages in isolation, higher level skills 
such as integrating and synthesizing information from multiple texts are typically not addressed 
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(although the paired passages on the National Assessment of Educational Progress [NAEP] 
Assessment, the SAT® verbal section, and some state assessments are an exception to this). 

In the modern world, it is rarely the case that all of the information needed to achieve a 
specific purpose is available within a single text. Scenario-based assessments, by contrast, 
incorporate a research-based model of literacy that assumes (a) readers have a specific question 
that they want to answer, (b) the answer to that question involves gathering information from 
multiple documents, and (c) comprehending that information involves integrating and 
synthesizing information in order to create what Perfetti, Rouet, and Britt (1999) called a 
documents model. 

The documents model describes the reader’s understanding of (a) the information 
presented in each individual text, (b) how the various texts are related, and (c) an interpretation 
or synthesis of the big picture implied by the collection of texts as a whole. Information that is 
consistent across documents is likely to be included in the documents model, while information 
that is discrepant is more likely to be excluded from the documents model. This type of 
document thinking is common in disciplines such as science and history (Goldman & Bisanz, 
2002; Wineburg, 1998). 

With the advent of technologies such as the Internet, the availability of information is 
now so great that it places new demands on readers. Documents on the Internet are written for 
different purposes and at different levels of quality. Readers in the 21st century must make sense 
of this information by combining it into meaningful and consistent documents models. 

Research on multiple document learning is sparse, particularly compared to the wealth of 
research available on how readers comprehend isolated texts (Goldman, 2004). The research that 
has been conducted, however, has shown that reader purpose has an effect on how well readers 
integrate information from multiple documents. Wiley and Voss (1996) found that students were 
more likely to create a coherent documents model that integrated information from multiple texts 
if they were asked to explain rather than describe how an event occurred. Similarly, Britt and 
Sommer (2004) found that reading purpose and reading strategies affect how well readers 
integrate information across texts (documents model). Specifically, the experimenters found that 
requiring students to summarize a text before reading a subsequent text improved the quality of 
their documents models. The experimenters also found that the quality of the documents model 
improved when students were asked to answer macrolevel questions (e.g., why questions) as 



9 




opposed to micro-level questions ( detail questions). Both findings support the idea that strategies 
directed at organizing and structuring texts also help readers to integrate information across 
multiple texts. 

In addition, Britt and Sommer (2004) found that explicitly asking students to integrate 
texts into a single story was much more effective at enhancing comprehension than asking 
students to read for the purposes of answering comprehension questions. This result provides 
evidence that not all readers spontaneously integrate multiple documents if they are not 
instructed to do so. In sum, promoting integration, either through explicit instruction or by 
fostering organization and structure strategies, helps to improve readers’ ability to integrate 
multiple texts. 

We acknowledge that the activity of integrating multiple texts to create a documents 
model is very demanding for young readers (Goldman & Bisanz, 2002). For instance, research 
has shown that high school students do not spontaneously integrate information from multiple 
sources (Britt, Goldman, & Perfetti, 1999). The results of Britt and Sommer (2004) indicated this 
is also frequently true of college students. However, as noted previously, when given the proper 
context and instruction, even extremely young children are able to connect information across 
multiple documents in meaningful ways (Brown & Campione, 1996; Rowe, 1994), and some 
researchers have argued that adolescence is the ideal time to introduce multiple-document 
thinking (Goldman, 2004). We believe that introducing younger children to meaningful and age- 
appropriate tasks that require the integration of multiple documents will better prepare them for 
the tasks they will be required to perform later in life. 

Although the previous discussion suggests that each assessment should include at least 
two related texts, it is also important to ensure that each text used in the assessment is long 
enough to have an identifiable structure. In an assessment context, a trade off exists between the 
number of texts (with an identifiable structure) that can be read and the amount of time available 
for testing. As the number of texts in an assessment increases, the length of any given text in the 
assessment must decrease due to time constraints. However, using short texts is problematic 
because texts of insufficient length have little or no identifiable text structure (i.e., a text’s 
organizational pattern), a factor that is critical for reading because it aids in the recall and 
comprehension of text (Gernsbacher, 1990; Meyer & Wijekumar, 2007; Ozuru, Rowe, O’Reilly, 
& McNamara, 2008; Williams, 2007). While the number of texts included in a single assessment 
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will depend upon the purpose stated in the scenario and the text genre, the total number of texts 
will probably have to be limited to about four. Accommodating more than four texts in a single 
assessment would likely require using texts that are too short and have no identifiable structure. 

An obvious, but important, additional design principle is that the texts included in the 
assessment must have some identifiable connection among them. The similarities among texts 
can include the discussion of similar topics, themes, principles, problems, descriptions, or critical 
analysis. Similarly, it is important that the texts included in each assessment be written by 
different authors and from different perspectives. Including texts that are too similar defeats the 
purpose of integration, which is to support cross-document thinking (and ultimately transfer of 
learning); if the documents are too similar, essentially they are a single document. Providing 
documents that are written from different perspectives may also help to promote a more liberal 
and sensitive attitude toward viewpoints held by different interest groups. 

Finally, integration and synthesis questions can take on at least two forms, one in which 
the goal of the reading is to synthesize the information from all documents to form a single 
understanding of the common message across different texts (i.e., documents model). 
Alternatively, integration and synthesis questions can be somewhat less demanding by requiring 
the reader to select portions of the various texts in order to achieve a specific goal. In the first 
instance, the task essentially requires the reader to produce a common understanding for all of 
the documents. In the second case, the task requires the reader to select portions from the various 
texts to answer a particular question or achieve a particular purpose; this type of task may not 
necessarily require the construction of a common documents model. In sum, each assessment 
must contain multiple documents selected to support intertextual thinking. 

Measure the Ability to Adopt a Critical Stance 

With the advent of the Internet and other technologies, people have a seemingly limitless 
amount of information at their finger tips. Because no uniform standards exist for posting 
information on the Internet, however, the information found there can be unsubstantiated and 
inaccurate (Fritch & Cromwell, 2002; Metzger, Flanagin, Eyal, Lemus, & McCann, 2003). Now, 
more than ever, people need to approach all forms of print and nonprint-based sources of 
information with a critical stance. A critical stance can be defined as a disposition or tendency to 
“presuppose that the quality of [all] information is potentially suspect and requires scrutiny with 
respect to its truth, relevance, and other dimensions of quality” (Graesser et al., 2007, p. 90). 
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After surveying the research on critical evaluation skills, Metzger (2007) identified five 
criteria that can be used to evaluate information (primarily for expository and persuasive texts): 
accuracy, authority, objectivity, currency, and coverage. Accuracy concerns the truth of the 
information: Is the information valid and error free? Authority refers to the credentials of the 
author: What are the author’s qualifications? Objectivity refers to the concept of bias and 
involves issues such as distinguishing fact from opinion, identifying conflicts of interest, and 
momentary gain: Is the author/source biased in his/her view? Currency refers to the time at 
which the information was published: Is the information out of date? Coverage refers to the 
amount of information contained in the document: Is there enough detail and depth? Is there 
missing information? 

While Metzger’s (2007) five-component model is useful in categorizing the important 
dimensions that readers should use in order to evaluate text, the model is incomplete because it 
does not consider the purpose of the reader. Accordingly, we suggest adding a sixth component 
to Metzger’s model that includes the concept of relevance. Relevance refers to the degree to 
which the text information is pertinent for achieving the reader’s specific goals: Is the 
information useful for answering the main question (Graesser, McNamara, & VanLehn, 2005; 
Graesser et al., 2007)? Such reflective, or metacognitive behavior is strategic and comprises an 
important part of learning (Azevedo & Cromley, 2004). 

Evaluation, like integration, is a difficult, high level competency. Research has shown 
that students frequently approach reading in an uncritical fashion (Brem, Russell, & Weems, 
2001; Greene, 1994; Vansledright & Kelly, 1998). For example, readers often fail to notice 
inconsistencies in text (Otero & Kintsch, 1992), fail to check for the accuracy of information 
(Flanagin & Metzger, 2000), and sometimes use a single criterion for making evaluations 
(Scholz-Crane, 1998). When evaluating online information, people will often use relatively 
superficial characteristics to make judgments of quality, such as the design and presentation of 
the Web site (Fogg et al., 2003). 

Training explicitly designed to improve critical thinking skills has been shown to be 
effective at promoting a critical stance and learning (Britt & Aglinskas, 2002; Sanchez, Wiley, & 
Goldman, 2006). Now, more than ever, evaluation skills are becoming more important as the 
flow of available information will only increase over time. 
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The assessment of evaluation and critical thinking skills is particularly important for 
assessments that include persuasive text. Persuasive texts should include a range of content that 
varies in terms of the six dimensions mentioned previously — accuracy, authority, objectivity, 
currency, coverage, and relevance. This may require adapting texts to highlight important 
distinctions (e.g., adapting some texts to increase credibility and others to decrease credibility). 
Questions can then be designed to tap into a student’s ability to evaluate texts based on the 
targeted dimensions. A similar approach can be employed to adapt expository texts when explicit 
signaling of potential problems is provided, and when it is developmentally and age appropriate. 
Questions should also be designed to ensure readers are effectively monitoring whether the 
information is relevant for their purpose. 

Measure the Ability to Ask and Answer Deep Questions 

Thus far, this section has focused on describing and providing the evidence base for three 
critical design features for the CBAL assessment: (a) providing a scenario that defines the 
purpose for reading, (b) providing multiple texts to facilitate integration and synthesis, and (c) 
providing texts that vary in terms of their quality to promote the use of evaluation skills. 
Collectively these three features are designed to promote deeper processing by encouraging 
readers to actively construct meaning from text (Singer, Graesser, & Trabasso, 1994). 
Unfortunately research suggests that readers often process text at a local and shallow level 
(McKoon & Ratcliff, 1992) and that such mindless processing can lead to an illusion of 
comprehension (Glenberg, Wilkinson, & Epstein, 1982; Langer, 1989; McNamara, Kintsch, 
Songer, & Kintsch, 1996). 

At least two reasons explain why students process text at a superficial and minimal level: 
the context and the text itself. First, research has shown that in many subject areas (e.g., history 
and social studies), the curriculum emphasizes memorizing lists of facts rather than focusing on 
the deeper causal mechanisms driving the sequence of events (Smith & Niemi, 2001; 
Vansledright, 1995). Second, this fact-driven instructional approach is also reflected in the 
structure of many textbooks. In an analysis of social studies texts. Beck, McKeown, and Gromoll 
(1989) found that the content and organization of the texts did not encourage deep thinking. For 
example, Beck at al. found that texts often present too much information with too little detail 
(i.e., mile- wide, inch-deep), contain sets of loosely connected statements, and have poor 
integration with previous sections. 
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Taken together, both a fact-driven curriculum and poorly written textbooks place 
challenging demands on students who strive to achieve deeper levels of understanding. In order 
to develop a coherent model for relatively incoherent text, students have to explain how and why 
the ideas in the text are related. The act of explaining the meaning of text has been referred to as 
the self-explanation effect (Chi, 2000; Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Chi, De 
Leeuw, Chiu, & LaVancher, 1994; McNamara, 2004; McNamara, O’Reilly, Best, & Ozuru, 
2006). Generating an explanation is an effortful activity in which readers construct meaning in 
order to draw inferences to fill in missing information, to make connections among ideas within 
the text, to integrate text information with existing knowledge, and to monitor and repair 
incorrect knowledge (Roy & Chi, 2005). 

Research has shown that students who self-explain the meaning of text are more likely to 
solve problems successfully, draw important inferences, build more consistent mental models, 
and develop a deeper understanding of the material in the text (Chi et al., 1989; Chi et al., 1994). 
Other work has indicated that the number and quality of explanations are related to the reader’s 
overall level of comprehension; students who produce more explanations, and students who 
produce higher quality explanations, learn more from text (Chi, 2000; McNamara et al., 2006). 
While some research suggests that high ability students produce higher quality and more 
sophisticated explanations than less skilled readers (Ozuru, Best, & McNamara, 2004), other 
work has indicated that specific training on how to self-explain primarily benefits lower but not 
higher ability students (McNamara, 2004; McNamara et al., 2006; O’Reilly, Best, & McNamara, 
2004). The important conclusion to draw from this work is that while the process of generating 
explanations may be natural for higher ability students, lower ability students can learn how to 
generate better explanations through training. 

In part, self-explanation is effective in promoting comprehension because it encourages 
readers to use their general knowledge to help fill in unstated relationships within the text 
(McNamara, 2004). In line with this reasoning, McNamara found that explicit training in self- 
explanation techniques helped reader’s comprehension only for more difficult low-cohesion text; 
the training did not help comprehension for easier, high-cohesion text in which the relationships 
among the concepts were more explicit. An alternative but not contradictory view is that the 
process of generating explanations helps readers to identify and repair gaps in then- 
understanding by making the process of comprehension more explicit and observable (see Chi, 
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2000, for a discussion of the incomplete text or repair view of self-explanation). In sum, the 
ability to generate explanations is a critical reading competency because it facilitates deeper and 
more complete understanding of text. 

Although the research base suggests that the ability to explain text is an important reading 
competency, measuring a student’s ability to do so poses significant challenges. First, generating 
self-explanations requires the reader to produce constructed responses. In some cases, the 
constructed responses can be lengthy, and as such, the production of extended responses shifts 
the construct more towards writing than reading. Because a separate CBAL writing strand 
already exists, requiring students to produce lengthy written explanations may not be 
appropriate. Second, scoring constructed-response explanations requires a considerable amount 
of human effort, including a significant amount of time devoted to training raters and scoring 
papers (Magliano et al, 2002; Magliano, Wiemer-Hastings, Millis, Munoz, & McNamara, 2005; 
McNamara, Levinstein, & Boonthum, 2004; McNamara et al., 2006; Millis, et al. 2004). While 
automated approaches for scoring self-explanations have met with some success, the level of 
accuracy achieved may not be sufficient for use in high stakes settings (McNamara et al., 2004). 

One possible solution to this problem is to create tasks and questions that load more 
heavily on carefully designed selected-response formats and, in some cases, short-answer 
constructed responses. Following the recommendations put forth in a recent Institute of 
Education Sciences (IES) report titled Organizing Instruction and Study to Improve Student 
Learning (Pashler et al., 2007), we advocate the design of assessment tasks that foster a student’s 
ability to construct explanations by both asking and answering deep questions. According to the 
panel’s recommendation, questions should focus on assessing causes, motivations, supporting 
evidence, planning, and logical justifications. Such evidence can be gathered by asking questions 
with the following stems (Pashler et al., 2007): Why? What caused X? How did X occur? What 
if, what if not? How does X compare to Y? What is the evidence for X? Why is X important? 

Similarly, question generation could be evaluated using the framework described in 
Graesser and Person (1994), which provided a 16-level classification system for evaluating the 
quality of questions. In this classification system, lower level questions are those that assess 
simple concepts such as verification and providing examples, and higher level questions are 
those that assess enablement and interpretation. In sum, we recommend assessing the construct 
of explanation by requiring students to both ask and answer deep questions. 
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Measure Basic Model Building Skill 

The skills discussed previously (i.e., integration, evaluation, and explanation) all belong 
to the CBAL category of applied comprehension skill. In terms of reading theory, applied 
comprehension skill is more closely associated with the situation model of representation 
because it requires the integration and application of the text information with existing 
knowledge (Kintsch, 1998). In this section, we discuss an additional collection of skills called 
model building skill. These skills are more in line with Kintsch’ s idea of a textbase representation 
that embodies the basic meaning, or gist, of a single text. 

It is useful to distinguish four specific model building skills: the ability to understand 
grade appropriate vocabulary; the ability to infer and generalize information; the ability to 
identify, locate, and retrieve important details; and the ability to extract the discourse structure 
(chunk, organize, summarize). These four skills are described as follows. 

Understanding grade appropriate vocabulary. It should be no surprise that knowing the 
meaning of the words in text is an important part of reading comprehension. Research has shown 
that a strong relationship exists between vocabulary and comprehension (Cunningham & 
Stanovich, 1997; Daneman, 1988; Hirsch, 2003). Correlations between comprehension and 
vocabulary measures are typically within the .6 -.7 range (Anderson & Freebody, 1981). While it 
is possible to comprehend a text without knowing the meaning of a few words, the role of 
vocabulary in comprehension should not be underestimated. For example, Nagy and Scott (2000) 
estimated that adequate reading comprehension depends on a person already knowing between 
90 and 95 percent of the words in a text. Moreover, differences in vocabulary size can magnify 
and cause problems over time. Students who become competent readers typically have much 
larger vocabularies than struggling readers in first grade, and this advantage tends to grow 
dramatically over time (Graves, Brunetti, & Slater, 1982; Hart & Risley, 1995). But what does it 
mean to know a word? It is important to note that knowing the meaning of a word takes place on 
many levels. Words have intricate meanings, including their connotative and denotative 
functions. As such, one must consider not only the number of words a person knows (breadth), 
but also the depth at which those words are known (Ouellet, 2006; Tannenbaum, Torgesen, & 
Wagner, 2006). In sum, vocabulary development is a critical part of learning to read well and 
appears to be a significant aspect of the gap between competent and struggling readers. While the 
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existence of this gap is discouraging, a number of studies have demonstrated that teaching words 
directly can improve comprehension (Beck & McKeown, 1991; Stahl & Fairbanks, 1986). 

Drawing inferences and making generalizations. The ability to draw inferences from text 
is one of the hallmarks of reading comprehension (McNamara & O’Reilly, in press). “An 
inference is defined as any piece of information that is not explicitly stated in the text” (McKoon 
& Ratcliff, 1992, p. 440). While a number of different types of inferences exists, the class of 
inferences that we are referring to here is more in line with the type of inferences required to 
build a coherent textbase, rather than the types of elaborative inferences that are characteristic of 
the construct of explanation (see Singer & Remillard, 2004). The ability to draw inferences is 
necessary because authors often fail to make the relations between concepts in the text explicit 
(Beck et al., 1989; Chi et al., 1994; Wilson & Anderson, 1986). As with vocabulary, the ability 
to draw inferences should not be understated, as it is one of the characteristic features that 
distinguish skilled from less skilled comprehenders (McNamara & O’Reilly, in press). 

For instance, skilled comprehenders make more appropriate inferences at both the local 
and global level. At the local level, less skilled comprehenders have difficultly resolving 
referential information such as anaphors (e.g., knowing that she refers to Mary, Oakhill & Yuill, 
1986; Yuill & Oakhill, 1988a). Less skilled comprehenders have increased difficulty with 
anaphor resolution when the distance between the antecedent and the referent increases (e.g., 
when the inference requires bridging over multiple sentences; Yuill & Oakhill, 1988a). 
Difficulties in relating information also extend to more global relations (i.e., generalizations). For 
example, less skilled comprehenders have difficulty relating successive topics in the text (Lorch, 
Lorch, & Morgan, 1987) and integrating information to construct the overall theme or main ideas 
of a passage (Palincsar & Brown, 1984; Stevens, 1988; Williams, 1986). On the other hand, 
skilled comprehenders are more likely to generate inferences that repair conceptual gaps in the 
text, whereas less skilled comprehenders are more likely to ignore gaps or fail to make the 
inferences to repair coherence gaps (Garnham, Oakhill, & Johnson-Laird, 1982; Long, Oppy, & 
Seely, 1994; Magliano & Millis, 2003; Magliano, Wiemer-Hastings, Millis, Munoz, & 
McNamara, 2002; Oakhill, 1984; Oakhill & Yuill, 1996; Oakhill, Yuill, & Donaldson, 1990; 
Yuill, Oakhill, & Parkin, 1989). Generating needed inferences is a complex process that can 
break down at a number of different levels (Cain, Oakhill, Barnes, & Bryant, 2001). While the 
process of generating inferences is complex, it appears that explicit training on how to generate 
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inferences is an effective method for promoting comprehension (McNamara, 2004; Stevens, 
1988; Williams, 1986; Yuill & Oakhill, 1988b). In short, the ability to draw inferences is a 
critical component of comprehension skill. 

Identify, locate, and retrieve important details (literal comprehension ). While the ability 
to infer information from text is an important reading competency, it is essential not to forget a 
more basic comprehension competency: the ability to understand the literal meaning of text. The 
literal meaning concerns the ideas, concepts, principles, events, dates, names, places, and people 
that are explicitly mentioned in the text; no inferences are necessary to understand literal 
information. In most comprehension assessments, students are allowed to reread the text while 
they are answering the comprehension questions. Under these circumstances, the ability to 
understand literal information essentially translates into the ability to locate and retrieve 
formation. 

According to Guthrie and Mosenthal (1987), searching for information in text may be 
viewed as a problem-solving activity that includes forming goals, selecting portions of the text, 
making a decision as to whether the selected information satisfies the goal, remembering the 
information, and repeating the process until the goal is met. Not surprisingly, research has shown 
that skilled comprehenders are more efficient and accurate than less skilled comprehenders at 
locating information in text (Cataldo & Oakhill, 2000). Cataldo and Oakhill found that good 
comprehenders often go directly to the target information while poor comprehenders are 
undirected in their search. The ability of good comprehenders to locate information was not due 
to their general spatial ability or their ability to remember the specific spatial locations of 
individual words. Rather, good comprehenders were more likely than less skilled comprehenders 
to remember the in-text sequence in which specific words appeared. In addition, the 
experimenters found that the performance of good comprehenders was more akin to the less 
skilled comprehenders when reading a scrambled text, suggesting that the good comprehenders 
took advantage of the coherent text structure to help them locate information. Together, these 
results suggest that the ability to locate information in text is dependent both on the structure of 
the text and the reader’s representation of the material. 

The idea of measuring vocabulary, inference, and literal information is not new to the 
assessment community. Most existing comprehension assessments measure most if not all of 
these constructs. Together, these constructs are meant to sample and approximate the reader’s 
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textbase representation of the material (the reader’s model building skill). However, 
approximating the textbase by only measuring its components is problematic. The textbase is an 
integrated representation of the text content that exists in the reader’s mind (Kintsch, 1998), and 
the whole is more than the sum of its parts. The point to be made is that if the goal is to measure 
the reader’s representation of the text, then a measure that more directly reflects the quality of a 
reader’s mental representations is needed. 

Extracting the discourse structure — The ability to chunk, summarize, and organize 
information. While measuring vocabulary, inference, and literal comprehension skills is 
important, it is also important to measure examinees’ performance relative to the skill of 
extracting the discourse structure. Two lines of research hold promise for facilitating this goal — 
summary writing and the work on text structure and graphical representations. 

A summary is a concise representation of the text content that includes an extraction of 
main ideas and important details and how they are related to each other. Research has shown that 
the act of producing summaries results in better comprehension (Moore, 1995), retention of text 
information (Rinehart, Stahl, & Erickson, 1986), and in some cases, better course performance 
(Radmacher & Latosi-Sawin, 1995). Summary writing has also been shown to result in increases 
in students’ ability to monitor their comprehension (Thiede & Anderson, 2003). 

Not surprisingly, research has revealed that the quality of student summaries varies as a 
function of age, experience, and ability (Gamer, 1985; Head, Readence, & Buss, 1989). For 
example, summaries written by less skilled comprehenders often lack consistency (McCutchen & 
Perfetti, 1982) and fail to adhere to the organizational structure of the text (Meyer, Brandt, & 
Bluth, 1980). Recent advances in automated scoring have enabled researchers to build programs 
that teach students how to improve their summary writing. One such system, called Summary 
Street, has been shown to improve both the quality of students’ summaries and their reading 
comprehension (Franzke, Kintsch, & Caccamise, 2005). 

Including the construct of summary writing in the CBAF reading assessment is 
problematic, however, because it requires the reader to construct a potentially lengthy written 
response. In other words, writing summaries in a constructed-response format may be more in 
line with the construct of writing than reading. This idea is supported by the fact that the quality 
of student summaries have been found to vary as a function of writing ability (Head et al., 1989). 
One possible solution to this problem is to assess summarization skills via a selected response 
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format. This approach is currently being used in the Test of English as a Foreign Language™ 
(TOEFL®). In particular, the TOEFL summarization task asks students to select appropriate 
summary statements from an option list that includes both correct and incorrect summary 
statements. Two types of incorrect summary statements are typically provided: statements that 
focus on minor details that are correct yet not appropriate for inclusion in a high level summary, 
and statements that represent erroneous inferences. 

Research has also shown that the quality of students’ summaries is frequently affected by 
structural characteristics such as headers. Topics that are signaled via headers are more likely to 
be included in student’s summaries, while topics that are not signaled are less likely to be 
included (Lorch, Pugzles-Lorch, Ritchey, McGovern, & Coleman, 2001). This leads us to our 
second possible approach for assessing the reader’s model of a text: the research on text structure 
and graphic representations. 

Text structure is the organizational pattern that logically connects the ideas and 
relationships within text (Williams, 2007). Both expository and narrative texts have identifiable 
structures. In narrative text, the structure is often referred to as story grammar and includes 
elements such as the setting, characters, plot, conflicts, and resolution (Mandler & Johnson, 
1977). Typical expository structures include comparison, problem and solution, cause and effect, 
sequence, description, and listing (Meyer & Wijekumar, 2007). Research has shown that texts 
with identifiable structures are easier to comprehend (Cataldo & Oakhill, 2000; Lorch et al., 
2001). Other work has revealed that skilled readers often use the structure of a text to aid 
comprehension and recall, but less skilled readers are less likely to do so (Meyer et al., 1980). 
Furthermore, interventions aimed at teaching students to identify text structure have been shown 
to be effective for improving comprehension (Meyer & Poon, 2001; Meyer & Wijekumar, 2007; 
Williams, 2007). In sum, text structure is an important aspect of comprehension and provides an 
additional way to capture what we mean by an integrated representation of the reader’ s textbase 
(i.e., discourse structure 4 ). Following, we briefly describe approaches for graphically 
representing the discourse structure. 

As mentioned previously, we have concerns with using extended constructed-response 
formats for measuring the discourse structure. One alternative is to use graphical representations 
that map out the structure. In the research and educational fields, the use of graphic 
representations to organize text has been given a variety of different names including graphic 
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organizers, concept maps, and knowledge maps (Griffin & Tulbert, 1995; Lambiotte, Skaggs, & 
Dansereau, 1993; Robinson & Skinner, 1996; Vitale & Romance, 2007). A graphic 
representation can be a figure, diagram, table, or chart that is used to depict the hierarchical, 
categorical, sequential, causal, and logical relationships among the ideas in text. Unlike a 
summary, graphic representations make minimal use of writing because the focus is more on 
representing the organization of the text’s important points rather than on presenting a coherent 
description of the information. Graphic representations allow the reader to quickly see the 
important ideas and how they relate, making them an economical way to represent the discourse 
structure (Robinson & Skinner, 1996). Indeed, research has shown that graphic representations 
can improve recall, comprehension, and concept acquisition, and also help to improve the ability 
to efficiently locate information in text (Griffin & Tulbert, 1995; Lambiotte et al., 1993; 
Robinson & Skinner, 1996; Vitale & Romance, 2007). Examples of the different types of graphic 
representations may be viewed at http://www.cast.org/publications/ncac/ncac go.html . In sum, 
we advocate measuring four components of model building: understanding key vocabulary, 
drawing necessary inferences, identifying important details, and extracting the discourse 
structure. 

Measure the Adequacy of Fundamental Reading Skills 

Thus far we have elaborated on the set of skills that is important for comprehension, but 
we have not yet discussed the collection of fundamental skills that enable students to read printed 
text. We refer to these fundamental reading skills as prerequisite reading skill. According to the 
simple view of reading (Gough & Tunmer, 1986; Hoover & Gough, 1990), reading 
comprehension depends on both language processes (e.g., model building and applied 
comprehension skill) and word recognition and decoding processes. Furthermore, the model 
states that both skills are necessary for reading comprehension and neither skill by itself is 
sufficient. While the simple view of reading was developed over 20 years ago, the basic premises 
of the model still hold today (Cutting & Scarborough, 2006; Vellutino, Tunmer, & Jaccard, 
2007). Research has shown that lower level reading s ki lls such as decoding contribute unique 
variance in standardized reading comprehension test scores over word recognition and language 
measures (Cunningham, Stanovitch, & Wilson, 1990; Hoover & Gough, 1990). However, as 
word recognition becomes more fluent and automatized, listening comprehension becomes a 
stronger predictor of reading ability, though word recognition continues to contribute significant 
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variance even in skilled readers (Carver & David, 2001; Gough & Walsh, 1991; Haenggi & 
Perfetti, 1994; McCormick, 1994). Nonetheless, weaknesses in fundamental reading skills such 
as word recognition and decoding can impair a reader’s ability to form an accurate model of a 
text (Perfetti, 1985). 

Why are lower level reading abilities such as word recognition and decoding important 
for reading comprehension? According to Perfetti’ s verbal efficiency theory (Perfetti, 1985), 
reading, like any other skill, is subject to resource limitations. In order to perform any complex 
skill, certain basic processes must be automatized to free up resources for more demanding 
higher level processes. When basic skills are lacking, they draw valuable resources from the 
higher level processes that demand them. 

In an extension of the verbal efficiency theory, Perfetti and Hart (2002) proposed the 
lexical quality hypothesis (LQH). According to this model, the quality of a word representation 
can vary on three levels: orthographic, phonetic, and semantic. The orthographic code specifies 
the word’s spelling, the phonetic code determines the pronunciation, and the semantic code 
provides the meaning. The central claim of the LQH is that if one or more representations of the 
word code are deficient, then the quality of the overall word representation decreases yielding 
subsequent decreases in both the efficiency and accuracy of word retrieval operations. 

Consequently, the LQH asserts that skilled and less skilled readers differ in two respects. 
First, skilled readers have more resources to repair or embellish impoverished representations. 
These resources may include more effective decoding, spelling, and grammatical skills. 
However, the key characteristic that distinguishes skilled from less skilled readers is the number 
of high quality word representations. Skilled readers are more efficient and accurate at retrieving 
words because the majority of their word representations are complete. This allows for fast and 
efficient retrieval of orthographic, phonemic, and semantic information. When word recognition 
and decoding become automatic, it frees up the resources needed for higher level processes such 
as inferencing (Cain, Oakhill, & Bryant, 2004; Daneman & Merikle, 1996; Perfetti, 1985). In 
other words, fast and efficient word identification skill is the foundation for text comprehension 
(Perfetti, 1985). 

In addition to word recognition and decoding ability, the research base also points to 
fluency as a crucial skill. Recent work has demonstrated that the ability to fluently read 
connected text (as measured by rate and accuracy) is correlated with reading comprehension 
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ability as measured by standardized reading comprehension assessments (Cutting & 

Scarborough, 2006; Jenkins, Fuchs, van den Broek, Espin, & Deno, 2003), state accountability 
assessments (McGlinchey & Hixon, 2004; Wiley & Deno, 2005), and NAEP (Daane, Campbell, 
Grigg, Goodman, & Oranje, 2005). In fact, Cutting and Scarborough found that a measure of 
reading fluency (rate) predicted 1% to 6% of additional variance of several standardized 
comprehension assessments over and above the effects of word recognition and language 
measures. The authors recommended modifying the simple view of reading (i.e., language and 
word recognition skill) to include fluency as a third component. 

The importance of prerequisite reading skills to comprehension, as evidenced by the 
research cited previously, supports our decision to include prerequisite skill in the CBAL reading 
competency model. The competency model currently includes three types of prerequisite reading 
skills: the ability to recognize familiar words, the ability to decode unfamiliar words, and the 
ability to read connected text fluently. Because reading connected text is a more integrated task, 
such tasks are the preferred format for CBAL summative assessments. Similarly, because 
measures of decoding and word recognition skills can facilitate analyses of readers’ underlying 
strengths and weaknesses, less integrated formats, such as asking students to read aloud from a 
list of pseudowords, are the preferred format for a more diagnostic or formative assessment. As a 
final note, it is important to reiterate that the construct of prerequisite reading skill refers to only 
the process of recognizing or decoding printed text; it does not refer to the general processes 
involved in comprehension. 5 

Recognize Technology ’s Impact: Reading in New Places 

Currently, many standardized and accountability assessments of reading use traditional 
forms of printed material to assess reading comprehension (e.g., excerpts from novels, textbooks, 
newspapers, magazines). However, in the new age of informationalism (Castells, 1996), students 
are frequently called upon to participate in reading activities that involve technology (Kirsch et 
al., 2007; Partnership for 21 st Century Skills, 2004, 2008; Warschauer, 2007). These activities 
include (but are not limited to) producing electronic documents, presentations, and 
communications (e.g., e-mail); performing information searches (e.g., Web browsing); and 
organizing electronic information. The use of technology is not limited to the work force, it 
extends into leisure time as well. More and more adolescents (and adults) are text messaging, 
reading blogs, engaging in online gaming and chat room discussions, watching videos (e.g., 
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YouTube) and making connections through on-line social networking environments (e.g., 
Facebook). For example, according to the Pew Internet and American Life Project, a large 
proportion of people between the ages of 12 and 49 frequently use the Internet on a regular basis 
and this trend is increasing over time (for a brief overview of the project findings see, Jones & 
Fox, 2009). In order to accommodate these changes, CBAL assessments will utili z e both 
traditional forms of printed material as well as the new forms of electronic media described 
previously. 



Summary and Conclusions 

This paper lays the foundation for a new approach for assessing reading comprehension 
in accountability settings. This new approach is called cognitively based assessment of, for and 
as learning (CBAL) because a cognitive model of domain competency provides the basis both 
for measuring learning and advancing learning. The approach involves first developing a 
cognitive model of reading competency and then using that model to design tasks and activities 
that provide high quality evidence about students’ mastery status on targeted skills. 

We began the discussion by emphasizing the need to test higher level reading skills such 
as integration/synthesis, evaluation/critical thinking, and explanation/reasoning. Collectively we 
refer to these skills as applied comprehension skill because they require readers to go beyond the 
basic meaning of a text in order to apply what has been learned in some meaningful way (e.g., 
solve a problem, make a decision). We outlined three reading activities in which applied 
comprehension skills are critically needed. The first activity involves constructing a document 
model to integrate and synthesize information from multiple texts. The second activity involves 
developing a critical stance in order to evaluate the quality of information. The third activity 
involves generating explanations both to develop deeper, more complete understandings of text 
content and to identify breakdowns in comprehension so that needed repairs can be made. A key 
aspect to the design involves presenting the assessment within the context of an authentic 
scenario designed to provide a realistic purpose for reading. Arguably, the construct of applied 
comprehension may go beyond general definitions of comprehension to include critical thinking, 
critical reasoning, decision making, and problem solving. 

While the applied comprehension skills discussed previously are critically needed to 
succeed at 21st century tasks, they are difficult to master and breakdowns can occur at many 
levels. Consequently, we also advocate measuring two additional sets of skills: model building 
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skill and prerequisite reading skill. Model building skill is the collection of abilities needed to 
extract the basic meaning, or gist, of a text. This set can be further broken down into four 
components: understanding the literal meaning of a text (e.g., locate/retrieve), understanding key 
vocabulary, drawing inferences/ making generalizations, and extracting the discourse structure 
(organize/ summarize). We proposed graphical task formats designed to measure all three of 
these model building skills in an integrated manner because they closely align with strategies 
designed to improve reading comprehension. We also mentioned that model building skill could 
occur in the context of printed text or nonprinted formats (e.g., spoken language). 

Because the ability to form an accurate mental model of a text depends on lower level 
reading skills, we also proposed several measures of prerequisite reading skill. In skilled readers, 
prerequisite reading skill requires little or no attention and resources. As a result, skilled readers 
are able to focus their attention and direct resources in ways that facilitate the application of 
higher level skills. On the other hand when lower level reading skills are not automated, valuable 
resources are no longer available for higher level processing. Consequently, we also advocated 
measuring three types of prerequisite reading skills: word recognition, decoding, and the ability 
to read text fluently. Prerequisite reading skill is bound to printed text and represents reading 
ability in its literal sense (unlike model building and applied comprehension, which can extend to 
forms of communication other than print). 

Taken together, applied model building and prerequisite reading skill define a complex 
reading construct. Assessments that measure all three levels of this construct provide a powerful 
tool for assessing reading development and helping students to succeed in the 21st century. 
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Notes 



*For example, one design might specify using four 90-minute summative assessments to be 
administered at four different time points throughout the year for a total of 360 minutes of 
testing time. As a result, each 90-minute assessment could be more focused and therefore 
assess the standards and necessary skills at a greater depth than what would be possible in a 
typical one-time year-end assessment. 

“ Appendix A of this report describes the skills branch of the competency model in more detail. 

Collectively, the skills branch has some overlap with Bloom’s Taxonomy that is often used in 
educational settings (Anderson & Krathwohl, 2001; Bloom, 1956). While differences exist in 
the models, Bloom’s notions of knowledge and comprehension roughly correspond to model 
building skill, while Bloom’s notions of application, analysis, synthesis, and evaluation 
roughly correspond to applied comprehension skill. A detailed comparison between Bloom’s 
taxonomy and the CBAL reading and writing competency models is found in Deane and 
O’Reilly’s work (2009). 

4 We prefer to use the term discourse structure as opposed to text structure to reflect the 

generative nature of comprehension. Text structure is often used to describe the features of the 
text. In contrast, discourse structure is intended to refer to the structure of the reader’s 
representation of the text. This is a very subtle but important difference. 

5 Although we recognize that the skills underlying higher level comprehension can also affect 

lower level processing such as word recognition. 

6 The individual competencies included in this set are the observable manifestations of the latent 

construct of automaticity, characterized cognitively as requiring minimal cognitive effort and 
attention. The performance appears as if a direct memory trace between spelling and meaning 
and/or pronunciation has been forged, though this latter theoretical claim is difficult to know 
or prove. 
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Appendix 

Cognitively Based Assessment of, for and as Learning (CBAL) 
Competency Model — The Skills Branch 



This section describes the skills branch of the CBAL reading competency model. It is 
broken down into three areas: prerequisite reading skill, model building skill, and applied 
comprehension skill. The top portion of Figure 1 is a graphical depiction of the skills branch of 
the CBAL reading competency. 



Prerequisite Reading Skill 

Prerequisite reading skill is the ability to accurately read text at a rate that is grade level 
appropriate. It includes the ability to recognize familiar words (e.g., house) and the ability to 
decode unfamiliar real or pseudo words. Prerequisite reading skill includes the following 
components: 

Recognize Common or Familiar Words 

This skill involves rapidly recognizing grade level appropriate familiar words. For 
students without disabilities, the skill involves recognizing words in printed or digital text. The 
recognize common or familiar words skill category differs from the other abilities listed in that 
only minimal processing effort is involved. 6 

Decode Unfamiliar Words or Pseudowords From Text 

This skill is the ability to produce plausible pronunciations of unfamiliar or pseudowords 
by applying English orthographic/phonetic conventions. 

Read Connected Text Fluently and With Appropriate Prosody 

This is the ability to read aloud at a rate and level of accuracy that is grade level 
appropriate. In some cases the construct may include the concept of prosody which is defined as 
the appropriate use of intonation, rhythm, focus, and emphasis where appropriate. 

Model Building Skill 

Model building skill concerns the reader’s ability to form a gist representation of the 
information contained within a single passage. The construct is broken down into four 
components: understanding the literal meaning (e.g., identifying important details, 
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locate/retrieve), understanding grade appropriate vocabulary, drawing necessary 
inferences/making key generalizations, and extracting the discourse structure (chunk, organize, 
summarize). 

Understanding the Literal Meaning of a Passage (Locate/Retrieve) 

This category concerns the ability to locate/retrieve, remember, and understand the 
details, facts, concepts, and individual sentences contained within a passage. This skill category 
also includes comprehending/remembering the gist of important sentences, and recognizing and 
producing accurate paraphrases of a sentence. Questions designed to measure mastery status 
relative to these skills will not require inferences or background knowledge. All of the necessary 
information required to answer literal questions is contained within the text. For the purposes of 
item writing, this construct is broken down into two components: the ability to locate and 
retrieve information, and the ability to produce and recognize accurate paraphrases of facts, 
concepts, and individual sentences contained within a passage. 

Understand Grade Appropriate Vocabulary 

This component includes the ability to acquire and use the type of technical vocabulary 
typically found in grade level appropriate text. It includes the ability to infer the meaning of 
unknown words from the contexts in which they are used and the ability to use acquired 
vocabulary in meaningful and appropriate ways. 

Infer and Generalize Information (Infer Relations Among Concepts or Text-Based Inferences ) 

This component concerns the ability to infer the relations among the concepts expressed 
within a single text. It includes the ability to make inferences across adjacent sentences, as well 
as the ability to make inferences across distal sentences. The construct also includes the ability to 
extract themes and main ideas from text. All the information required to answer this item type is 
provided within the text. No special background knowledge is necessary. As a general rule, items 
that require the use of two or more sentences are considered inference items, and items that 
require the use of background knowledge are explanation based or explanatory inference items 
(i.e., an applied comprehension construct). There are two basic types of inference items: local 
and global inference. The simplest form of a local inference is the anaphoric type. Anaphoric 
inferences involve making connections among local ideas in text by using discourse markers 
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(e.g., he, she, they, it, latter, second) to connect ideas across sentences. Another, more 
demanding type of local inference requires the reader to make inferences that extend the explicit 
situation depicted in the text. The second basic type of inference is the global or thematic 
inference. This type of inference requires the reader to make generalizations based on large 
portions of text. 

Extract and Use Discourse Structure (Chunk, Organize, Summarize) 

This component concerns the ability to formulate a network representation of how the 
main ideas and supporting details are related to each other. Discourse structure involves 
identifying the complex spatial, temporal, and causal relationships that summarize and classify 
the key relationships (often hierarchical) among the core ideas in text. For literary texts, 
discourse structure involves identifying the relationship between characters, plot, setting, 
conflicts, and resolutions, and distinguishing chronological sequence from the narrative 
sequence. For expository texts, discourse structure involves classifying, listing, describing, 
sequencing, comparing, and contrasting concepts; identifying the problem and solutions; and 
identifying cause and effect relationships. By definition, discourse structure is an amalgamation 
of both literal and inferential information that results in a succinct, orderly synopsis of the 
concepts and the relationships among ideas in text. For some texts the discourse structure is 
relatively explicit and straightforward. For other texts the structure is implicit and has to be 
inferred. 



Applied Comprehension Skill 
Evaluate and Critique ( Critical Reading/Thinking) 

This component concerns the ability to deploy critical thinking skills to evaluate text. 
Evaluation involves judging, ranking, and appraising the value of a text as a whole and the 
specific features that comprise it. It also involves verifying the truth value of text contents where 
appropriate (e.g., if the student has adequate knowledge of the topic). In narrative text, it 
involves the ability to evaluate author craft. In persuasive text, it involves the ability to critically 
evaluate author/ source creditability, claims, support for claims and conclusions, implications of 
claims and conclusions, and how current the information is. It includes the ability to detect the 
author’s purpose for writing, position on the issue, potential bias, logical flaws in the arguments, 
and the ability to identify conflicting arguments. It also includes the ability to determine how 
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relevant an argument or document is for the issue at hand and the how relevant it is for achieving 
the reader’s overall goal. In short, evaluation involves discerning the quality of text content as 
defined by the six dimensions mentioned in the prior review: accuracy, authority, objectivity, 
currency, coverage, and relevance. 

Integrate and Synthesize With Other Text 

This component concerns the ability to understand, integrate, and synthesize information 
from multiple documents. It is the ability to manage multiple documents by combining 
information from each document to produce a coherent, meaningful, and systematic document 
model of the text content (i.e., integrate). In other words, integrating requires the reader to 
identify and combine information from several documents that is relevant for achieving his or 
her overall purpose for reading. It also involves the ability to synthesize, which is defined as the 
ability to generate new ideas, concepts by combining the information from two or more texts in 
novel ways. While constructing a document model requires the use of similar skills involved in 
building a model of a single text (model building), the process involved in creating a document 
model is much more complex. Within a single text, there is often an explicit or strongly implied 
general model of the text; that is, the ideas contained within a single text should be logically 
connected and the text as a whole should be coherent. This is because the author wrote the text 
for a particular purpose and audience. In contrast, a document model requires the reader to notice 
connections, draw comparisons, and make contrasts between two or more texts that are often 
written by different authors and for different purposes. Such differences make the process of 
document building much more difficult because the links among the documents are not explicitly 
provided, and the reader has to generate and infer the links based on the reader’s purpose for 
reading the collection of documents. The construct also involves not only linking documents, but 
also the ability to apply the principles and concepts learned in one or more documents to 
hypothesize, extrapolate, and solve problems contained in related texts. This application of core 
ideas or transfer of knowledge is one of the hallmarks of expert performance. 

Generate Explanations — The Ability to Reason and Explain Text 

This component concerns the ability to construct a deep and consistent understanding of 
text by generating explanations. The construct includes the ability to draw inferences to fill in 
missing information, to make connections among ideas that are not easily inferable, to integrate 
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text information with existing knowledge, and to monitor and repair incorrect knowledge. 
Generating explanations involves the ability to use general background knowledge to elaborate, 
explain, reason, or infer unstated relationships within text. Explaining involves the ability to 
monitor comprehension for the purposes of identifying inconsistencies in text and also to identify 
misconceptions in understanding. Generating explanations often involves asking and answering 
deep questions that focus on causes, motivations, supporting evidence, planning, and logical 
justifications. Such questions can take on many generic forms, such as the following: Why? 

What caused X? How did X occur? What if, what if not? How does X compare to Y? What is the 
evidence for X? Why is X important? Generating explanatory inferences is different from the 
draw inferences competency included under the model building skill (i.e., text-based inferences). 
Generating explanations requires the use of general knowledge and reasoning processes to 
connect concepts, while text-based inferences can be made on the basis of the information 
contained solely within the text. 
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